Segmentation Granularity in Dependency Representations for Korean

Previous work on Korean language processing has proposed different basic segmentation units. This paper explores different possible dependency representations for Korean using different levels of segmentation granularity — that is, different schemes for morphological segmentation of tokens into syntactic words. We provide a new Universal Dependencies (UD)-like corpus based on different levels of segmentation granularity for Korean. The corpus contains 67K words in 5,000 sentences which are split into training, development and evaluation data sets. We report parsing results using the new dependency corpus for Korean and compare them with the previous Korean UD corpus. 1 Dependency Parsing and the Korean Language Language processing including morphological analysis for Korean has traditionally been based on the eojeol, which is a basic segmentation unit delimited by a blank in the sentence. Let us consider the sentence in (1), which contains ten eojoels (the corresponding morphological analysis is found in Figure 1). The number of eojoels is entirely based on the blank space character and the tenth eojeol in (1) also includes the punctuation mark. Almost all natural-language processing systems that have been previously developed for Korean have used the eojeol as a fundamental unit of analysis. As Korean is an agglutinative language, joining content and functional morphemes is very productive and they can be combined exponentially. For example, yeoghal (‘role’) is a content morpheme (a common noun) and -eul, a case marker (‘ACC’, accusative), is a functional morpheme.1 They form together a single eojeol yeoghal-eul (‘role + ACC’). A predicate gangjoha-ass-da (‘focused’) also consists of the content morpheme gangjo-ha (‘focus’) and its functional morphemes, -ass (‘PAST’, past tense) and da (‘IND’, indicative), respectively. In this paper, we analyze different levels of segmentation granularity in dependency representations for syntactic annotation (§2). We then propose a scheme to build a new Universal Dependencies (UD)-like corpus for Korean based on segmentation granularity (§3). UD has been developed cross-linguistically using a consistent treebank annotation scheme for many languages.2 We provide 5,000 sentences based on each of the segmentation granularity possibilities described in this paper. We also present its UD parsing results, compare them with previously proposed UD for Korean (§4), and discuss future perspectives of dependency annotation and parsing for Korean (§5). 2 Segmentation Granularity for Korean We define the following four different levels of segmentation granularity for Korean. These granularity levels have been independently proposed in previous work on Korean language processing as different basic segmentation units.